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The use of a 1.9-cm radio sextant capable of precise; tracking of the sun has produced 
accurate measurements of total atmospheric microwave refraction. These data arc used to 
verify the high correlation of such refraction with surface refractivity for low altitude angles. 
The values of the correlation coefficients obtained vary from 92.2 percent at L6 degrees to 
98 percent at 2 degrees. An empirical predictor is developed, based on this correlation, 
which satisfactorily accounts for the observed refraction. The mathematical form of the 
predictor is given, and suggestions are made for its use. 



1. Introduction 

Theoretical investigations conducted by B. R. 
Bean and B. A. Cahoon [1957] have indicated that 
total atmospheric microwave refraction can be pre- 
dicted from observations of surface refractivity. 
Specifically, their proposed method of prediction re- 
quires that the total bending angle be linearly 
correlated with surface refractivity. That is, total 
atmospheric microwave refraction (r) can be calcu- 
lated from an equation of the form 

T = b N s +a (1) 

where N s is surface refractivity and b and a are 
coefficients which are functions of observed altitude 
(elevation) angle. 

The objectives of the analysis described in this 
article were to verify by experiment the utility of 
the prediction technique suggested by Bean and 
Cahoon for discrete low altitude angles and to extend 
their analysis by developing suitable empirical 
formulas for the evaluation of the parameters a and 
b for arbitrary values of altitude angle. 

This experiment was made possible by the recent 
development at Collins Eadio Company of a 1.9-cm 
radio sextant capable of precise tracking of the sun. 
With this instrument, total atmospheric microwave 
refraction can be measured. The total bending 
angle is given by the difference between the observed 
position of the sun as determined by the radio 
sextant and the true position as derived from the 
solar ephemeris. 

Thus, by simultaneously measuring the apparent 
altitude angle of the sun and the surface refractivity, 
it is possible to verify the linear correlation of refrac- 
tion angle with surface refractivity. 

2. Collection and Preparation of Data 

Solar tracking data obtained from the radio sex- 
tant during the period from August through Decem- 



ber 1959 at a site near Cedar Rapids, Iowa, were 
available for the analysis. Most of the available 
data for altitude angles less than 20 degrees were 
used. Exceptions involved rejection of data when 
independent information indicated a malfunction of 
the equipment or rejection on the basib of inter- 
mittent or sparse low angle data. The data selected 
for further analysis included 35 sunsets and 14 
sunrises. 

The data for each of these 49 cases were corrected 
for predictable equipment errors. Corrections were 
made for Inductosyn gain-phase error and outer-loop 
bias. The dial index error, as determined from 
i-e fraction-corrected, high-angle tracking data for the 
same day, was removed from the data for each sunrise 
and sunset [Anway, 1961]. 

The remaining difference between observed and 
true solar altitude angle was assumed to consist 
entirely of atmospheric refraction plus a random 
tracking error. In order to eliminate the random 
component and to permit evaluation of measured 
refraction at specific values of observed altitude 
angle, a polynomial was fitted by the method of 
least squares to each of the 49 refraction plots. 
Generally, a fifth-order polynomial was required to 
obtain a satisfactory fit over the pertinent range of 
observed altitude angles. In a few instances the 
polynomial was judged to be a poor representation 
of the initial refraction plot over a portion of its 
range because of gaps in the initial data or similar 
difficulties. These segments were rejected with as 
much objectivity as possible by assigning applica- 
bility ranges to the polynomials prior to further 
analysis. One sunset was eliminated completely in 
this manner, leaving 34 sunsets and 14 sunrises for 
further analysis. 

It was assumed at this point that the polynomials, 
evaluated at any given altitude angle, would yield 
the best estimates of refraction, exclusive of any 
short-term refraction fluctuations which would have 
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been smoothed out along' with the random error. 
Hereafter, reference to the experimental values of 
refraction is understood to mean reference to the 
polynomial estimate of refraction. Figure 1 shows 
the mean and standard deviation of measured refrac- 
tion at integral altitude angles from 2 to 16 degrees. 
Values of surface refractivity were computed using 
the Smith and Weintraub equation [1953] which is 
valid throughout the microwave region. 
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In (2), n is the refractive index, e is the partial 
pressure of water vapor in millibars, p is the total 
pressure in millibars, and T is the absolute tempera- 
ture in degrees Kelvin. 

Surface pressure, temperature, and wet bulb de- 
pression normally were measured every one-half hour 
during solar tracking. Thus, for each sunrise or 
sunset it was possible to associate a computed sur- 
face refractivity with each observed altitude angle. 
Because the changes in N s for successive observa- 
tions generally were quite small, no interpolations 
between observations were considered necessary. 

3. Correlation 

Plotting the measured refraction against the asso- 
ciated surface refractivity at selected values of alti- 
tude angle produced scatter diagrams such as those 
shown in figure 2. A regression analysis was per- 
formed on the data of the scatter diagram for each 
integral altitude angle from 2 to 16 degrees. The 
intercept a and slope b for each regression line and 
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Figure 1. Mean and standard deviation of measured refraction 



other pertinent statistics are tabulated in table 1. 

The table indicates that the number of points used 
to determine each regression line varies from a mini- 
mum of 21 at an angle of 2 degrees to a maximum 
of 47 at angles of 13 degrees and 14 degrees. The 
smaller number of points at the lower values of alti- 
tude angle results from a scarcity of tracking data at 
those angles. In particular, no sunrise data w r ere 
available below an observed altitude angle of about 
5 degrees. On the other hand, at altitude angles of 
13 degrees and 14 degrees, all but one of the 48 cases 
considered were applicable. 

(Comments on the nature of the slopes and inter- 
cepts of the regression lines are deferred to the next 
section on the development of predictors.) 

Without a correlation with N s , a prediction of re- 
fraction equal to the mean refraction listed for each 
angle would result in the corresponding standard de- 
viation or prediction uncertainty in the next column. 
Employment of the regression line of slope b and 
intercept a, however, results in the standard devia- 
tion shown in the final column. The uncertainty is 
seen to be reduced in this manner by a factor varying 
roughly between 0.2 and 0.4. 

It is not implied that the relatively constant 
standard deviation about the regression lines for 
angles of 10 degrees or greater is the true uncertainty 
of refraction prediction in this region. It is probable 
that this standard deviation is the accuracy limit of 
the analysis techniques employed. Nonrandom 
components in the original error plots, discrepancies 
in the curve-fitting process, and errors in the estima- 
tion of surface refractivity certainly contribute 
significantly to this lower limit of the standard 
deviation about the prediction line. 
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Figure 2. Scatter diagrams of measured refraction versus 
measured surface refractivity for selected observed altitude 
angles. 
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The values of the standard deviations about the 
prediction or regression lines are plotted in figure 3 
along with four values obtained by Bean and 
Cahoon in their analysis of calculated refraction 
errors. Direct comparison may be made at 15 
degrees and 3 degrees, where the theoretical values 
are respectively 38 percent and 55 percent of the 
experimental values. This discrepancy is explained 
by the assumed accuracy limit of the analysis. 
Consequently, in the vicinity of 2 degrees, where the 
standard deviation of refraction exceeds the accuracy 
limit, agreement is markedly improved. 

Furthermore, it is significant that in the Bean and 
Cahoon analysis the correlation coefficient con- 
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Figure 3. Standard deviation of measured refraction about 
the regression lines versus observed altitude angle. 



tinuously increased with altitude angle, while (he 
values in table 1 reach a maximum at 3 degrees and 
4 degrees. This disagreement also may be explained 
by the presence of the analysis accuracy limit, since, 
wit J i a constant standard deviation about the 
regression line, the correlation coefficient must 
reduce as the slope of the line reduces. 

4. Prediction 

The correlation demonstrated in the previous 
section is of considerable interest in itself, but the 
application of this correlation in the prediction or 
estimation of pointing errors produced by refraction 
is the anticipated result of greatest general interest. 

The tabulated regression line slopes and intercepts 
do not constitute a very convenient formula for 
prediction. It is desirable to have a continuous em- 
pirical function that adequately reproduces the re- 
gression line prediction accuracy at the discrete 
observed altitude angles. An example of this type 
of empirical predictor is developed in the following 
paragraphs. 

The examined predictor is of the form 



T=bN,+a 



a=T 



(ho+Bf 



(1) 
(3) 



6=(^X10--)[oot»o- 1 ^} (4) 

Where r is the total refraction or bending angle in 
degrees, and a and b are functions of the observed 
altitude angle (h ) with the dimensions of degrees 
and degrees per surface refractivity unit respectively. 
The parameters A, B, C, D, E, and F are positive 
constants to be determined empirically. It is noted 
that as h Q becomes large, a approaches zero and b 
approaches the product of a constant times the 
cotangent of the observed altitude angle (or the 
tangent of the observed zenith angle). 

Defining Ab by: 



Ab=b- 



-(^XIO" 6 ) cot h 



(5) 



permits careful examination of small departures of b 
from the simple cotangent form which is entirely 
adequate at higher observed altitude angles. 

When the experimental values of a and b, as deter- 
mined in the regression analysis, are plotted against 
observed altitude angle as in figures 4 and 5, it is 
observed that they oscillate about what might be 
termed a smooth curve. Further, the oscillations 
are compensatory, in the sense that positive excur- 
sions in a which produce increased r values tend to 
be accompanied by negative excursions in Ab which 
produce decreased r values. (See (1) and (5).) Al- 
though the cause of these oscillations is not readily 
identifiable, it will be demonstrated later that, what- 
ever their cause, they do little to improve prediction 
accuracy as compared to smoothed predictors. The 
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applicable values of a quoted by Bean and Cahoon 
also are shown in figure 4. Fitting (3) to these 
three values results in the expression 

-40 



a= JJ^+2W (6) 

This expression produces the solid curve of figure 4 ; 
the dashed curve represents a fit of (3) to data 
derived from a supplemental ray-tracing analysis of 
model atmospheres in a manner similar to that em- 
ployed by Bean and Cahoon. However, the solid 
curve not only produces a superior fit to the experi- 
mental values, but also permits direct comparison 
of b values with those of Bean and Cahoon. 

Therefore, after defining the intercepts at integral 
values of h by (6), the slopes were redetermined by 
the method of least-squares. The corresponding re- 
determined b values are shown in figure 5. It is 
apparent immediately that the initial oscillations 
have almost disappeared, and that the agreement 
with Bean and Cahoon data is excellent. 

Fitting the redetermined b values with the em- 
pirical expression (4) results in the parameters D, E, 
and F shown in the first column of table 2. The 
second column of the same table indicates the param- 
eters required for a good fit to the three Bean and 
Cahoon points of figure 5, and the third column 
displays similar parameters for the supplemental ray- 
tracing results. Also shown in the table are the 
approximate mean values of surface refractivity for 
the three data groups. 
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Figure 4. Intercept of regression line versus observed altitude 
angle. 



i.o I 1 1 1 1 r 



-2.0 



1 r 






§ B © © © 8 



• EXPERIMENTAL 
© REDETERMINED 
O BEAN AND CAHOON 



6 8 10 12 

OBSERVED ALTITUDE ANGLE , deg 



Figure 5. Slope of regression line minus (180X 10~ 6 ) cot Hq/tt 
versus observed altitude angle. 

Table 2 





RED 


BC 


M 




(Redeter- 


(Bean and 


(Model) 




mined) 


Cahoon) 




D 


45.6 


42.5 


43.0 


E 


0.4 


0.4 


0.4 


F 


2.64 


2.64 


2.69 


Ns 


325 to 330* 


334 


347 



*In the experiment, the statistics of Ns depend upon the observed altitude 
angle. 

A comparison of refraction predicted by the three 
columns of table 2 and refraction predicted by the 
regression lines of table 1 (EXP) is given in figure 6. 
For an assumed N s of 325, the value of r RED from the 
first column of table 2 has been subtracted from the 
refraction predicted by each of the remaining three 
over the range of observed altitude angles from 2 to 
16 degrees. It is apparent that the combined effect 
of the oscillations noted in the experimental values 
of a and b results in prediction discrepancies of less 
than 0.0012 degree (about 4 seconds of arc) when 
compared with the empirical predictor, r RED at 
2V S =325. It also is evident that r RED satisfactorily 
averages this oscillatory effect. 

It is interesting to note that r RED , t bc and r M 
exhibit increasingly larger values at each observed 
altitude angle. This increase is accounted for by 
an increase in the slope, b, because the intercept, 
a, is defined by (6) for each of the three empirical 
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Figure 6. Prediction differences versus observed altitude angle. 

predictors. It is reasonable to expect that the 
exact form of the predictor should depend upon the 
corresponding mean value of surface refractivity. 

A final evaluation of the predictor r RED is illus- 
trated in table 3. This table compares the standard 
deviation about the original regression line (as in 
figure 3) with that about the empirical predictor 
tred at each observed altitude angle. It is seen that 
substitution of the empirical predictor for the origi- 
nal regression lines results in only slight degradation 
in prediction accuracy. 

Table 3 
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2 


31.1 


31.1 
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20.7 


22.0 


1.3 
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18.4 


18.7 
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5. Conclusions 

The use of a precision radio sextant to measure 
total atmospheric refraction has permitted an ex- 
perimental verification of the prediction technique 
suggested by Bean and Cahoon. It has proven the 
feasibility of improved refraction estimation when 
only surface meteorological conditions are known. 

Although the precision of the measurement tech- 
nique and subsequent analysis was not sufficient to 
obtain exact agreement with theory, both the posi- 
tion of the theoretical regression lines and their cor- 
relation coefficients have been verified substantially. 
Further refinements in experimental technique should 
produce improved agreement with theory. 

Fitting polynomials to the original refraction data 
should have smoothed refraction fluctuations ade- 
quately with periods less than one-half hour. The 
surface refractivity for a given one-half hour segment 
of each polynomial was estimated from single meas- 
urements of meteorological parameters. Thus, 
measured refraction values were smoothed without 
a similar smoothing of surface refractivity determi- 
nations. Even so, a significant correlation was 
demonstrated with discrete, uniformly spaced refrac- 
tivity samples. 

Jt also has been shown that relatively simple em- 
pirical expressions may be used to evaluate param- 
eters a and b of (1) at observed altitude angles be- 
tween 2 and 16 degrees. The nature of the empirical 
ex pressions is such that negligible' errors are produced 
if (1) is employed in the range from 16 to 90 degrees. 

It is suggested that for maximum accuracy, the 
parameters of (3) and/or (4) should be adjusted ac- 
cording to the mean value of surface refractivity en- 
countered in a given location during a given period. 
The predictor best suited to the experimental data 
presented here is not proposed for general use. Until 
the nature of this suggested dependence is defined 
more carefully, the Bean and Cahoon parameters of 
table 2 probably are more suitable for general pre- 
diction. 
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